IEEE Spectrum is the flagship publication of the IEEE — the world’s largest professional organization devoted to engineering and applied sciences. Our articles, podcasts, and infographics inform our readers about developments in technology, engineering, and science.
IEEE websites place cookies on your device to give you the best user experience. By using our websites, you agree to the placement of these cookies. To learn more, read our Privacy Policy.
Stay ahead of the latest technology trends. Become an IEEE member.
Enjoy more free content and benefits by creating an account
Saving articles to read later requires an IEEE Spectrum account
The Institute content is only available for members
Downloading full PDF issues is exclusive for IEEE Members
Access to Spectrum's Digital Edition is exclusive for IEEE Members
Following topics is a feature exclusive for IEEE Members
Adding your response to an article requires an IEEE Spectrum account
Create an account to access more content and features on IEEE Spectrum, including the ability to save articles to read later, download Spectrum Collections, and participate in conversations with readers and editors. For more exclusive content and features, consider Joining IEEE.
This article is for IEEE members only. Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →
Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, archives, PDF downloads, and other benefits. Learn more →
Illustration: IEEE Spectrum. Jeff Bezos: Ted S. Warren/AP; Venky Harinarayan: David Paul Morris/Bloomberg/Getty Images; Patent: US Patent and Trademark Office.
The history of AI is often told as the story of machines getting smarter over time. What’s lost is the human element in the narrative, how intelligent machines are designed, trained, and powered by human minds and bodies.
In this six-part series, we explore that human history of AI—how innovators, thinkers, workers, and sometimes hucksters have created algorithms that can replicate human thought and behavior (or at least appear to). While it can be exciting to be swept up by the idea of superintelligent computers that have no need for human input, the true history of smart machines shows that our AI is only as good as we are.
Part 6: Mechanical Turk Revisited
At the turn of millennium, Amazon began expanding its services beyond bookselling. As the variety of products on the site grew, the company had to figure out new ways to categorize and organize them. Part of this task was removing tens of thousands of duplicate products that were popping up on the website.
Engineers at the company tried to write software that could automatically eliminate all duplicates across the site. Identifying and deleting objects seemed to be a simple task, one well within the capacities of a machine. Yet the engineers soon gave up, describing the data-processing challenge as “insurmountable.” This task, which presupposed the ability to notice subtle differences and similarities between pictures and text, actually required human intelligence.
Amazon was left with a conundrum. Deleting duplicate products from the site was a trivial task for humans, but the sheer number of duplicates would require a huge workforce. Coordinating that many workers on one task was not a trivial problem.
An Amazon manager named Venky Harinarayan came up with a solution. His patent described a “hybrid machine/human computing arrangement,” which would break down tasks into small units, or “subtasks” and distribute them to a network of human workers.
The Amazon manager’s patent described a “hybrid machine/human computing arrangement.”
In the case of deleting duplicates, a central computer could divide Amazon’s site into small sections—say, 100 product pages for can openers—and send the sections to human workers over the Internet. The workers could then identify duplicates in these small units and send their pieces of the puzzle back.
This distributed system offered a crucial advantage: The workers didn’t have to be centralized in one place but could instead complete the subtasks on their own personal computers wherever they happened to be, whenever they chose. Essentially, what Harinaryran developed was an effective way to distribute low-skill yet difficult-to-automate work to a broad network of humans who could work in parallel.
The method proved so effective in Amazon’s internal operations, Jeff Bezos decided this system could be sold as a service to other companies. Bezos turned Harinaryan’s technology into a marketplace for laborers. There, businesses that had tasks that were easy for humans (but hard to automate) could be matched with a network of freelance workers, who would do the tasks for small amounts of money.
Thus was born Amazon Mechanical Turk, or mTurk for short. The service launched in 2005, and the user base quickly grew. Businesses and researchers around the globe began uploading thousands of so-called “human intelligence tasks” onto the platform, such as transcribing audio or captioning images. These tasks were dutifully carried out by an internationally dispersed and anonymous group of workers for a small fee (one aggrieved worker reported an average fee of 20 cents per task).
The name of this new service was a wink at the chess-playing machine of the 18th century, the Mechanical Turk invented by the huckster Wolfgang von Kempelen. And just like that faux automaton, inside which hid a human chess player, the mTurk platform was designed to make human labor invisible. Workers on the platform are not represented with names, but with numbers, and communication between the requester and the worker is entirely depersonalized. Bezos himself has called these dehumanized workers “artificial artificial intelligence.”
A 2017 paper found that Turkers earn a median wage of approximately US $2 per hour, and only 4 percent earned more than $7.25 per hour.
Today, mTurk is a thriving marketplace with hundreds of thousands of workers around the world. While the online platform provides a source of income for people who otherwise might not have access to jobs, the labor conditions are highly questionable. Some critics have argued that by keeping the workers invisible and atomized, Amazon has made it easier for them to be exploited. A research paper [PDF] published in December 2017 found that workers earned a median wage of approximately US $2 per hour, and only 4 percent earned more than $7.25 per hour.
Interestingly, mTurk has also become crucial for the development of machine-learning applications. In machine learning, an AI program is given a large data set, then learns on its own how to find patterns and draw conclusions. MTurk workers are frequently used to build and label these training data sets, yet their role in machine learning is often overlooked.
The dynamic now playing out between the AI community and mTurk is one that has been ever-present throughout the history of machine intelligence. We eagerly admire the visage of the autonomous “intelligent machine,” while ignoring, or even actively concealing, the human labor that makes it possible.
Perhaps we can take a lesson from the author Edgar Allan Poe. When he viewed von Kempelen’s Mechanical Turk, he was not fooled by the illusion. Instead, he wondered what it would be like for the chess player trapped inside, the concealed laborer “tightly compressed” among cogs and levers in “exceedingly painful and awkward positions.”
In our current moment, when headlines about AI breakthroughs populate our news feeds, it’s important to remember Poe’s forensic attitude. It can be entertaining—if sometimes alarming—to be swept up in the hype over AI, and to be carried away by the vision of machines that have no need for mere mortals. But if you look closer, you’ll likely see the traces of human labor.
The HX-63 cipher machine is an electromechanical, rotor-based system designed and built by Crypto AG. The machine uses nine rotors [center right] to encrypt messages. A dual paper-tape printer is at the upper left.
PETER ADAMS
Growing up in New York City, I always wanted to be a spy. But when I graduated from college in January 1968, the Cold War and Vietnam War were raging, and spying seemed like a risky career choice. So I became an electrical engineer, working on real-time spectrum analyzers for a U.S. defense contractor.
In 1976, during a visit to the Polish Army Museum in Warsaw, I saw an Enigma, the famous German World War II cipher machine. I was fascinated. Some years later, I had the good fortune of visiting the huge headquarters of the cipher machine company Crypto AG (CAG), in Steinhausen, Switzerland, and befriending a high-level cryptographer there. My friend gave me an internal history of the company written by its founder,
Boris Hagelin. It mentioned a 1963 cipher machine, the HX-63.
Like the Enigma, the HX-63 was an electromechanical cipher system known as a rotor machine. It was the only electromechanical rotor machine ever built by CAG, and it was much more advanced and secure than even the famous Enigmas. In fact, it was arguably the most secure rotor machine ever built. I longed to get my hands on one, but I doubted I ever would.
Fast forward to 2010. I'm in a dingy third subbasement at a French military communications base. Accompanied by two-star generals and communications officers, I enter a secured room filled with ancient military radios and cipher machines. Voilà! I am amazed to see a Crypto AG HX-63, unrecognized for decades and consigned to a dusty, dimly lit shelf.
I carefully extract the 16-kilogram (35-pound) machine. There's a hand crank on the right side, enabling the machine to operate away from mains power. As I cautiously turn it, while typing on the mechanical keyboard, the nine rotors advance, and embossed printing wheels feebly strike a paper tape. I decided on the spot to do everything in my power to find an HX-63 that I could restore to working order.
If you've never heard of the HX-63 until just now, don't feel bad. Most professional cryptographers have never heard of it. Yet it was so secure that its invention alarmed William Friedman, one of the greatest cryptanalysts ever and, in the early 1950s, the first chief cryptologist of the U.S. National Security Agency (NSA). After reading a 1957 Hagelin patent (more on that later), Friedman realized that the HX-63, then under development, was, if anything, more secure than the NSA's own KL-7, then considered unbreakable. During the Cold War, the NSA built thousands of KL-7s, which were used by every U.S. military, diplomatic, and intelligence agency from 1952 to 1968.
The reasons for Friedman's anxiety are easy enough to understand. The
HX-63 had about 10600 possible key combinations; in modern terms, that's equivalent to a 2,000-bit binary key. For comparison, the Advanced Encryption Standard, which is used today to protect sensitive information in government, banking, and many other sectors, typically uses a 128- or a 256-bit key.
In the center of the cast-aluminum base of the HX-63 cipher machine is a precision Swiss-made direct-current gear motor. Also visible is the power supply [lower right] and the function switch [left], which is used to select the operating mode—for example, encryption or decryption.Peter Adams
A total of 12 different rotors are available for the HX-63, of which nine are used at any one time. Current flows into one of 41 gold-plated contacts on the smaller-diameter side of the rotor, through a conductor inside the rotor, out through a gold-plated contact on the other side, and then into the next rotor. The incrementing of each rotor is programmed by setting pins, which are just visible in the horizontal rotor.
Peter Adams
Just as worrisome was that CAG was a privately owned Swiss company, selling to any government, business, or individual. At the NSA, Friedman's job was to ensure that the U.S. government had access to the sensitive, encrypted communications of all governments and threats worldwide. But traffic encrypted by the HX-63 would be unbreakable.
Friedman and Hagelin were good friends. During World War II, Friedman had helped make Hagelin a very wealthy man by suggesting changes to one of Hagelin's cipher machines, which paved the way for the U.S. Army to license Hagelin's patents. The resulting machine, the
M-209-B, became a workhorse during the war, with some 140,000 units fielded. During the 1950s, Friedman and Hagelin's close relationship led to a series of understandings collectively known as a “gentleman's agreement" between U.S. intelligence and the Swiss company. Hagelin agreed not to sell his most secure machines to countries specified by U.S. intelligence, which also got secret access to Crypto's machines, plans, sales records, and other data.
But in 1963, CAG started to market the HX-63, and Friedman became even more alarmed. He convinced Hagelin not to manufacture the new device, even though the machine had taken more than a decade to design and only about 15 had been built, most of them for the French army. However, 1963 was an interesting year in cryptography. Machine encryption was approaching a crossroads; it was starting to become clear that the future belonged to electronic encipherment. Even a great rotor machine like the HX-63 would soon be obsolete.
That was a challenge for CAG, which had never built an electronic cipher machine. Perhaps partly because of this, in 1966, the relationship among CAG, the NSA, and the CIA went to the next level. That year, the NSA delivered to its Swiss partner an electronic enciphering system that became the basis of a CAG machine called the H-460. Introduced in 1970, the machine was a failure. However, there were bigger changes afoot at CAG: That same year, the CIA and the German Federal Intelligence Service secretly acquired CAG for US $5.75 million. (Also in 1970, Hagelin's son Bo, who was the company's sales manager for the Americas and who had opposed the transaction,
died in a car crash near Washington, D.C.)
Although the H-460 was a failure, it was succeeded by a machine called the H-4605, of which thousands were sold. The H-4605 was designed with NSA assistance. To generate random numbers, it used multiple shift registers based on the then-emerging technology of CMOS electronics. These numbers were not true random numbers, which never repeat, but rather pseudorandom numbers, which are generated by a mathematical algorithm from an initial “seed."
This mathematical algorithm was created by the NSA, which could therefore decrypt any messages enciphered by the machine. In common parlance, the machines were “backdoored." This was the start of a new era for CAG. From then on, its electronic machines, such as the HC-500 series, were secretly designed by the NSA, sometimes with the help of corporate partners such as Motorola. This U.S.-Swiss operation was code-named
Rubicon. The backdooring of all CAG machines continued until 2018, when the company was liquidated.
William F. Friedman [top] dominated U.S. cryptology for over 40 years and was the first chief cryptologist of the U.S. National Security Agency. His friend Boris Hagelin [bottom], a brilliant Swedish inventor and entrepreneur, founded Crypto AG in 1952 in Zug, Switzerland, and built it into the world's largest cipher-machine company.TOP, U.S. ARMY; BOTTOM: CRYPTO AG
Parts of this story emerged in leaks by CAG employees before 2018 and, especially, in a subsequent investigation by the
Washington Post and a pair of European broadcasters,
Zweites Deutsches Fernsehen, in Germany, and Schweizer Radio und Fernsehen, in Switzerland. The Post's article, published on 11 February 2020, touched off firestorms in the fields of cryptology, information security, and intelligence.
The revelations badly damaged the Swiss reputation for discretion and dependability. They triggered civil and criminal litigation and an investigation by the Swiss government and, just this past May, led to the
resignation of the Swiss intelligence chief Jean-Philippe Gaudin, who had fallen out with the defense minister over how the revelations had been handled. In fact, there's an interesting parallel to our modern era, in which backdoors are increasingly common and the FBI and other U.S. intelligence and law-enforcement agencies sporadically tussle with smartphone manufacturers over access to encrypted data on the phones.
Even before these revelations, I was deeply fascinated by the HX-63, the last of the great rotor machines. So I could scarcely believe my good fortune in 2020 when, after years of negotiations, I took possession of an HX-63 for my research for the
Association des Réservistes du Chiffre et de la Sécurité de l'Information, a Paris-based professional organization of cryptographers and information-security specialists. This particular unit, different from the one I had seen a decade before, had been untouched since 1963. I immediately began to plan the restoration of this historically resonant machine.
People have been using codes and ciphers to protect sensitive information for a couple of thousand years. The first ciphers were based on hand calculations and tables. In 1467, a mechanical device that became known as the Alberti cipher wheel was introduced. Then, just after World War I, an enormous breakthrough occurred, one of the greatest in cryptographic history: Edward Hebern in the United States, Hugo Koch in the Netherlands, and Arthur Scherbius in Germany, within months of one another, patented electromechanical machines that used rotors to encipher messages. Thus began the era of the rotor machine. Scherbius's machine became the basis for the famous Enigma used by the German military from the 1930s until the end of WW II.
To understand how a
rotor machine works, first recall the basic goal of cryptography: substituting each of the letters in a message, called plaintext, with other letters in order to produce an unreadable message, called ciphertext. It's not enough to make the same substitution every time—replacing every F with a Q, for example, and every K with an H. Such a monoalphabetic cipher would be easily solved.
PENETRATING THE ENIGMA
A simple cipher machine, such as the Enigma machine used by the German Army during World War II, has three rotors, each with 26 positions. Each position corresponds to a letter of the alphabet. Electric current enters at a position on one side of the first rotor, corresponding to a letter, say
T. The current travels through two other rotors in the same way and then, finally, exits the third rotor at a position that corresponds to a different letter, say R. So in this case, the letter T has been encrypted as R. The next time the operator strikes a key, one or more of the rotors move with respect to one another, so the next letter is encrypted with an entirely different set of permutations. In the Enigma cipher machines [below] a plugboard added a fixed scramble to the encipherment of the rotors, swapping up to 13 letter pairs.
A rotor machine gets around that problem using—you guessed it—rotors. Start with a round disk that's roughly the diameter of a hockey puck, but thinner. On both sides of the disk, spaced evenly around the edge, are 26 metal contacts, each corresponding to a letter of the English alphabet. Inside the disk are wires connecting a contact on one side of the disk to a different one on the other side. The disk is connected electrically to a typewriter-like keyboard. When a user hits a key on the keyboard, say
W, electric current flows to the W position on one side of the rotor. The current goes through a wire in the rotor and comes out at another position, say L. However, after that keystroke, the rotor rotates one or more positions. So the next time the user hits the W key, the letter will be encrypted not as L but rather as some other letter.
Though more challenging than simple substitution, such a basic, one-rotor machine would be child's play for a trained cryptanalyst to solve. So rotor machines used multiple rotors. Versions of the Enigma, for example, had either three rotors or four. In operation, each rotor moved at varying intervals with respect to the others: A keystroke could move one rotor or two, or all of them. Operators further complicated the encryption scheme by choosing from an assortment of rotors, each wired differently, to insert in their machine. Military Enigma machines also had a plugboard, which swapped specific pairs of letters both at the keyboard input and at the output lamps.
The rotor-machine era finally ended around 1970, with the advent of electronic and software encryption, although a Soviet rotor machine called
Fialka was deployed well into the 1980s.
The HX-63 pushed the envelope of cryptography. For starters it has a bank of nine removable rotors. There's also a “modificator," an array of 41 rotary switches, each with 41 positions, that, like the plugboard on the Enigma, add another layer, an unchanging scramble, to the encryption. The unit I acquired has a cast-aluminum base, a power supply, a motor drive, a mechanical keyboard, and a paper-tape printer designed to display both the input text and either the enciphered or deciphered text. A function-control switch on the base switches among four modes: off, “clear" (test), encryption, and decryption.
In encryption mode, the operator types in the plaintext, and the encrypted message is printed out on the paper tape. Each plaintext letter typed into the keyboard is scrambled according to the many permutations of the rotor bank and modificator to yield the ciphertext letter. In decryption mode, the process is reversed. The user types in the encrypted message, and both the original and decrypted message are printed, character by character and side by side, on the paper tape.
While encrypting or decrypting a message, the HX-63 prints both the original and the encrypted message on paper tape. The blue wheels are made of an absorbent foam that soaks up ink and applies it to the embossed print wheels.Peter Adams
Beneath the nine rotors on the HX-63 are nine keys that unlock each rotor to set the initial rotor position before starting a message. That initial position is an important component of the cryptographic key.Peter Adams
To begin encrypting a message, you select nine rotors (out of 12) and set up the rotor pins that determine the stepping motion of the rotors relative to one another. Then you place the rotors in the machine in a specific order from right to left, and set each rotor in a specific starting position. Finally, you set each of the 41 modificator switches to a previously determined position. To decrypt the message, those same rotors and settings, along with those of the modificator, must be re-created in the receiver's identical machine. All of these positions, wirings, and settings of the rotors and of the modificator are collectively known as the key.
The HX-63 includes, in addition to the hand crank, a nickel-cadmium battery to run the rotor circuit and printer if no mains power is available. A 12-volt DC linear power supply runs the motor and printer and charges the battery. The precision 12-volt motor runs continuously, driving the rotors and the printer shaft through a reduction gear and a clutch. Pressing a key on the keyboard releases a mechanical stop, so the gear drive propels the machine through a single cycle, turning the shaft, which advances the rotors and prints a character.
The printer has two embossed alphabet wheels, which rotate on each keystroke and are stopped at the desired letter by four solenoids and ratchet mechanisms. Fed by output from the rotor bank and keyboard, mechanical shaft encoders sense the position of the alphabet printing wheels and stop the rotation at the required letter. Each alphabet wheel has its own encoder. One set prints the input on the left half of the paper tape; the other prints the output on the right side of the tape. After an alphabet wheel is stopped, a cam releases a print hammer, which strikes the paper tape against the embossed letter. At the last step the motor advances the paper tape, completing the cycle, and the machine is ready for the next letter.
As I began restoring the HX-63, I quickly realized the scope of the challenge. The plastic gears and rubber parts had deteriorated, to the point where the mechanical stress of motor-driven operation could easily destroy them. Replacement parts don't exist, so I had to build such parts myself.
After cleaning and lubricating the machine, I struck a few keys on the keyboard. I was delighted to see that all nine cipher rotors turned and the machine printed a few characters on the paper tape. But the printout was intermittently blank and distorted. I replaced the corroded nickel-cadmium battery and rewired the power transformer, then gradually applied AC power. To my amazement, the motor, rotors, and the printer worked for a few keystrokes. But suddenly there was a crash of gnashing gears, and broken plastic bits flew out of the machine. Printing stopped altogether, and my heartbeat nearly did too.
I decided to disassemble the HX-63 into modules: The rotor bank lifted off, then the printer. The base contains the keyboard, power supply, and controls. Deep inside the printer were four plastic “snubbers," which cushion and position the levers that stop the ratchet wheels at the indicated letter. These snubbers had disintegrated. Also, the foam disks that ink the alphabet wheels were decomposing, and gooey bits were clogging the alphabet wheels.
I made some happy, serendipitous finds. To rebuild the broken printer parts, I needed a dense rubber tube. I discovered that a widely available neoprene vacuum hose worked perfectly. Using a drill press and a steel rod as a mandrel, I cut the hose into precise, 10-millimeter sections. But the space deep within the printer, where the plastic snubbers are supposed to be, was blocked by many shafts and levers, which seemed too risky to remove and replace. So I used right-angle long-nosed pliers and dental tools to maneuver the new snubbers under the mechanism. After hours of deft surgery, I managed to install the snubbers.
WHEELS WITHIN WHEELS: WHY THE HX-63 IS SO SECURE
The HX-63 has nine rotors
and also uses a technique
called reinjection. Each
rotor has a set of conductors
that connect each and
every electrical contact
on one side of the rotor
with a different contact on
the other side. For every
rotor the pattern of these
connections is unique. When
the operator strikes a key on
the keyboard, representing
one of 26 letters, current
travels through the set of
nine rotors twice, once in
each direction, and then
through a separate set of
15 rotor contacts at least
two times. This reinjection
technique greatly increases
the complexity of the cipher.
The ink wheels were made of an unusual porous foam. I tested many replacement materials, settling finally on a dense blue foam cylinder. Alas, it had a smooth, closed-cell surface that would not absorb ink, so I abraded the surface with rough sandpaper.
After a few more such fixes, I faced just one more snafu: a bad paper-tape jam. I had loaded a new roll of paper tape, but I did not realize that this roll had a slightly smaller core. The tape seized, tore, and jammed under the alphabet wheels, deeply buried and inaccessible. I was stymied—but then made a wonderful discovery. The HX-63 came with thin stainless-steel strips with serrated edges designed specifically to extract jammed paper tape. I finally cleared the jam, and the restoration was complete.
One of the reasons why the HX-63 was so fiendishly secure was a technique called reinjection, which increased its security exponentially. Rotors typically have a position for each letter of the alphabet they're designed to encrypt. So a typical rotor for English would have 26 positions. But the HX-63's rotors have 41 positions. That's because reinjection (also called reentry) uses extra circuit paths beyond those for the letters of the alphabet. In the HX-63, there are 15 additional paths.
Here's how reinjection worked in the HX-63. In encryption mode, current travels in one direction through all the rotors, each introducing a unique permutation. After exiting the last rotor, the current loops back through that same rotor to travel back through all the rotors in the opposite direction. However, as the current travels back through the rotors, it follows a different route, through the 15 additional circuit paths set aside for this purpose. The exact path depends not only on the wiring of the rotors but also on the positions of the 41 modificators. So the total number of possible circuit configurations is 26! x 15!, which equals about 5.2 x 10
38. And each of the nine rotors' internal connections can be rewired in 26! different ways. In addition, the incrementing of the rotors is controlled by a series of 41 mechanical pins. Put it all together and the total number of different key combinations is around 10600.
Such a complex cipher was not only unbreakable in the 1960s, it would be extremely difficult to crack even today. Reinjection was first used on the NSA's KL-7 rotor machine. The technique was invented during WW II by Albert W. Small, at the U.S. Army's Signal Intelligence Service. It was the subject of a secret patent that Small filed in 1944 and that was finally granted in 1961 (No.
2,984,700).
Meanwhile, in 1953, Hagelin applied for a U.S. patent for the technique, which he intended to use in what became the HX-63. Perhaps surprisingly, given that the technique was already the subject of a patent application by Small, Hagelin was granted his patent in 1957 (No.
2,802,047). Friedman, for his part, had been alarmed all along by Hagelin's use of reinjection, because the technique had been used in a whole series of vitally important U.S. cipher machines, and because it was a great threat to the NSA's ability to listen to government and military message traffic at will.
The series of meetings between Friedman and Hagelin that resulted in the cancellation of the HX-63 was mentioned in a 1977 biography of Friedman,
The Man Who Broke Purple, by Ronald Clark, and it was further detailed in 2014 through a disclosure by the NSA's
William F. Friedman Collection.
After a career as an electrical engineer and inventor, author Jon D. Paul now researches, writes, and lectures on the history of digital technology, especially encryption. In the 1970s he began collecting vintage electronic instruments, such as the Tektronix oscilloscopes and Hewlett-Packard spectrum analyzers seen here.
Peter Adams
The revelation of Crypto AG's secret deals with U.S. intelligence may have caused a bitter scandal, but viewed from another angle, Rubicon was also one of the
most successful espionage operations in history—and a forerunner of modern backdoors. Nowadays, it's not just intelligence agencies that are exploiting backdoors and eavesdropping on “secure" messages and transactions. Windows 10's “telemetry" function continuously monitors a user's activity and data. Nor are Apple Macs safe. Malware that allowed attackers to take control of a Mac has circulated from time to time; a notable example was Backdoor.MAC.Eleanor, around 2016. And in late 2020, the cybersecurity company FireEye disclosed that malware had opened up a backdoor in the SolarWinds Orion platform, used in supply-chain and government servers. The malware, called SUNBURST, was the first of a series of malware attacks on Orion. The full extent of the damage is still unknown.
The HX-63 machine I restored now works about as well as it did in 1963. I have yet to tire of the teletype-like motor sound and the clack-clack of the keyboard. Although I never realized my adolescent dream of being a secret agent, I am delighted by this little glimmer of that long-ago, glamorous world.
And there's even a postscript. I recently discovered that my contact at Crypto AG, whom I'll call “C," was also a security officer at the Swiss intelligence agencies. And so for decades, while working at the top levels of Crypto AG, “C" was a back channel to the CIA and Swiss intelligence agencies, and even had a CIA code name. My wry old Swiss friend had known everything all along!
This article appears in the September 2021 print issue as “The Last Rotor Machine."
To Probe Further
The Crypto AG affair was described in a pair of Swedish books. One of them was
Borisprojektet : århundradets största spionkupp : NSA och ett svensk snille lurade en hel värld [translation: The Boris Project: The Biggest Spy Coup of the Century: NSA and a Swedish genius cheated an entire world], 2016, Sixten Svensson, Vaktelförlag, ISBN 978-91-982180-8-4.
Also, in 2020, Swiss editor and author Res Strehle published
Verschlüsselt: Der Fall Hans Bühler [translation: Encrypted: The Hans Bühler Case], and later Operation Crypto. Die Schweiz im Dienst von CIA und BND [Operation Crypto: Switzerland in the Service of the CIA and BND].
Like the Enigma, the HX-63 was an electromechanical cipher system known as a rotor machine. It was the only electromechanical rotor machine ever built by CAG, and it was much more advanced and secure than even the famous Enigmas. In fact, it was arguably the most secure rotor machine ever built. I longed to get my hands on one, but I doubted I ever would.
Fast forward to 2010. I'm in a dingy third subbasement at a French military communications base. Accompanied by two-star generals and communications officers, I enter a secured room filled with ancient military radios and cipher machines. Voilà! I am amazed to see a Crypto AG HX-63, unrecognized for decades and consigned to a dusty, dimly lit shelf.
I carefully extract the 16-kilogram (35-pound) machine. There's a hand crank on the right side, enabling the machine to operate away from mains power. As I cautiously turn it, while typing on the mechanical keyboard, the nine rotors advance, and embossed printing wheels feebly strike a paper tape. I decided on the spot to do everything in my power to find an HX-63 that I could restore to working order.
If you've never heard of the HX-63 until just now, don't feel bad. Most professional cryptographers have never heard of it. Yet it was so secure that its invention alarmed William Friedman, one of the greatest cryptanalysts ever and, in the early 1950s, the first chief cryptologist of the U.S. National Security Agency (NSA). After reading a 1957 Hagelin patent (more on that later), Friedman realized that the HX-63, then under development, was, if anything, more secure than the NSA's own KL-7, then considered unbreakable. During the Cold War, the NSA built thousands of KL-7s, which were used by every U.S. military, diplomatic, and intelligence agency from 1952 to 1968.
The reasons for Friedman's anxiety are easy enough to understand. The
HX-63 had about 10600 possible key combinations; in modern terms, that's equivalent to a 2,000-bit binary key. For comparison, the Advanced Encryption Standard, which is used today to protect sensitive information in government, banking, and many other sectors, typically uses a 128- or a 256-bit key.
In the center of the cast-aluminum base of the HX-63 cipher machine is a precision Swiss-made direct-current gear motor. Also visible is the power supply [lower right] and the function switch [left], which is used to select the operating mode—for example, encryption or decryption.Peter Adams
A total of 12 different rotors are available for the HX-63, of which nine are used at any one time. Current flows into one of 41 gold-plated contacts on the smaller-diameter side of the rotor, through a conductor inside the rotor, out through a gold-plated contact on the other side, and then into the next rotor. The incrementing of each rotor is programmed by setting pins, which are just visible in the horizontal rotor.
Peter Adams
Just as worrisome was that CAG was a privately owned Swiss company, selling to any government, business, or individual. At the NSA, Friedman's job was to ensure that the U.S. government had access to the sensitive, encrypted communications of all governments and threats worldwide. But traffic encrypted by the HX-63 would be unbreakable.
Friedman and Hagelin were good friends. During World War II, Friedman had helped make Hagelin a very wealthy man by suggesting changes to one of Hagelin's cipher machines, which paved the way for the U.S. Army to license Hagelin's patents. The resulting machine, the
M-209-B, became a workhorse during the war, with some 140,000 units fielded. During the 1950s, Friedman and Hagelin's close relationship led to a series of understandings collectively known as a “gentleman's agreement" between U.S. intelligence and the Swiss company. Hagelin agreed not to sell his most secure machines to countries specified by U.S. intelligence, which also got secret access to Crypto's machines, plans, sales records, and other data.
But in 1963, CAG started to market the HX-63, and Friedman became even more alarmed. He convinced Hagelin not to manufacture the new device, even though the machine had taken more than a decade to design and only about 15 had been built, most of them for the French army. However, 1963 was an interesting year in cryptography. Machine encryption was approaching a crossroads; it was starting to become clear that the future belonged to electronic encipherment. Even a great rotor machine like the HX-63 would soon be obsolete.
That was a challenge for CAG, which had never built an electronic cipher machine. Perhaps partly because of this, in 1966, the relationship among CAG, the NSA, and the CIA went to the next level. That year, the NSA delivered to its Swiss partner an electronic enciphering system that became the basis of a CAG machine called the H-460. Introduced in 1970, the machine was a failure. However, there were bigger changes afoot at CAG: That same year, the CIA and the German Federal Intelligence Service secretly acquired CAG for US $5.75 million. (Also in 1970, Hagelin's son Bo, who was the company's sales manager for the Americas and who had opposed the transaction,
died in a car crash near Washington, D.C.)
Although the H-460 was a failure, it was succeeded by a machine called the H-4605, of which thousands were sold. The H-4605 was designed with NSA assistance. To generate random numbers, it used multiple shift registers based on the then-emerging technology of CMOS electronics. These numbers were not true random numbers, which never repeat, but rather pseudorandom numbers, which are generated by a mathematical algorithm from an initial “seed."
This mathematical algorithm was created by the NSA, which could therefore decrypt any messages enciphered by the machine. In common parlance, the machines were “backdoored." This was the start of a new era for CAG. From then on, its electronic machines, such as the HC-500 series, were secretly designed by the NSA, sometimes with the help of corporate partners such as Motorola. This U.S.-Swiss operation was code-named
Rubicon. The backdooring of all CAG machines continued until 2018, when the company was liquidated.
William F. Friedman [top] dominated U.S. cryptology for over 40 years and was the first chief cryptologist of the U.S. National Security Agency. His friend Boris Hagelin [bottom], a brilliant Swedish inventor and entrepreneur, founded Crypto AG in 1952 in Zug, Switzerland, and built it into the world's largest cipher-machine company.TOP, U.S. ARMY; BOTTOM: CRYPTO AG
Parts of this story emerged in leaks by CAG employees before 2018 and, especially, in a subsequent investigation by the
Washington Post and a pair of European broadcasters,
Zweites Deutsches Fernsehen, in Germany, and Schweizer Radio und Fernsehen, in Switzerland. The Post's article, published on 11 February 2020, touched off firestorms in the fields of cryptology, information security, and intelligence.
The revelations badly damaged the Swiss reputation for discretion and dependability. They triggered civil and criminal litigation and an investigation by the Swiss government and, just this past May, led to the
resignation of the Swiss intelligence chief Jean-Philippe Gaudin, who had fallen out with the defense minister over how the revelations had been handled. In fact, there's an interesting parallel to our modern era, in which backdoors are increasingly common and the FBI and other U.S. intelligence and law-enforcement agencies sporadically tussle with smartphone manufacturers over access to encrypted data on the phones.
Even before these revelations, I was deeply fascinated by the HX-63, the last of the great rotor machines. So I could scarcely believe my good fortune in 2020 when, after years of negotiations, I took possession of an HX-63 for my research for the
Association des Réservistes du Chiffre et de la Sécurité de l'Information, a Paris-based professional organization of cryptographers and information-security specialists. This particular unit, different from the one I had seen a decade before, had been untouched since 1963. I immediately began to plan the restoration of this historically resonant machine.
People have been using codes and ciphers to protect sensitive information for a couple of thousand years. The first ciphers were based on hand calculations and tables. In 1467, a mechanical device that became known as the Alberti cipher wheel was introduced. Then, just after World War I, an enormous breakthrough occurred, one of the greatest in cryptographic history: Edward Hebern in the United States, Hugo Koch in the Netherlands, and Arthur Scherbius in Germany, within months of one another, patented electromechanical machines that used rotors to encipher messages. Thus began the era of the rotor machine. Scherbius's machine became the basis for the famous Enigma used by the German military from the 1930s until the end of WW II.
To understand how a
rotor machine works, first recall the basic goal of cryptography: substituting each of the letters in a message, called plaintext, with other letters in order to produce an unreadable message, called ciphertext. It's not enough to make the same substitution every time—replacing every F with a Q, for example, and every K with an H. Such a monoalphabetic cipher would be easily solved.
PENETRATING THE ENIGMA
A simple cipher machine, such as the Enigma machine used by the German Army during World War II, has three rotors, each with 26 positions. Each position corresponds to a letter of the alphabet. Electric current enters at a position on one side of the first rotor, corresponding to a letter, say
T. The current travels through two other rotors in the same way and then, finally, exits the third rotor at a position that corresponds to a different letter, say R. So in this case, the letter T has been encrypted as R. The next time the operator strikes a key, one or more of the rotors move with respect to one another, so the next letter is encrypted with an entirely different set of permutations. In the Enigma cipher machines [below] a plugboard added a fixed scramble to the encipherment of the rotors, swapping up to 13 letter pairs.
Chris Philpot; SOURCE: JON D. PAUL
A rotor machine gets around that problem using—you guessed it—rotors. Start with a round disk that's roughly the diameter of a hockey puck, but thinner. On both sides of the disk, spaced evenly around the edge, are 26 metal contacts, each corresponding to a letter of the English alphabet. Inside the disk are wires connecting a contact on one side of the disk to a different one on the other side. The disk is connected electrically to a typewriter-like keyboard. When a user hits a key on the keyboard, say
W, electric current flows to the W position on one side of the rotor. The current goes through a wire in the rotor and comes out at another position, say L. However, after that keystroke, the rotor rotates one or more positions. So the next time the user hits the W key, the letter will be encrypted not as L but rather as some other letter.
Though more challenging than simple substitution, such a basic, one-rotor machine would be child's play for a trained cryptanalyst to solve. So rotor machines used multiple rotors. Versions of the Enigma, for example, had either three rotors or four. In operation, each rotor moved at varying intervals with respect to the others: A keystroke could move one rotor or two, or all of them. Operators further complicated the encryption scheme by choosing from an assortment of rotors, each wired differently, to insert in their machine. Military Enigma machines also had a plugboard, which swapped specific pairs of letters both at the keyboard input and at the output lamps.
The rotor-machine era finally ended around 1970, with the advent of electronic and software encryption, although a Soviet rotor machine called
Fialka was deployed well into the 1980s.
The HX-63 pushed the envelope of cryptography. For starters it has a bank of nine removable rotors. There's also a “modificator," an array of 41 rotary switches, each with 41 positions, that, like the plugboard on the Enigma, add another layer, an unchanging scramble, to the encryption. The unit I acquired has a cast-aluminum base, a power supply, a motor drive, a mechanical keyboard, and a paper-tape printer designed to display both the input text and either the enciphered or deciphered text. A function-control switch on the base switches among four modes: off, “clear" (test), encryption, and decryption.
In encryption mode, the operator types in the plaintext, and the encrypted message is printed out on the paper tape. Each plaintext letter typed into the keyboard is scrambled according to the many permutations of the rotor bank and modificator to yield the ciphertext letter. In decryption mode, the process is reversed. The user types in the encrypted message, and both the original and decrypted message are printed, character by character and side by side, on the paper tape.
While encrypting or decrypting a message, the HX-63 prints both the original and the encrypted message on paper tape. The blue wheels are made of an absorbent foam that soaks up ink and applies it to the embossed print wheels.Peter Adams
Beneath the nine rotors on the HX-63 are nine keys that unlock each rotor to set the initial rotor position before starting a message. That initial position is an important component of the cryptographic key.Peter Adams
To begin encrypting a message, you select nine rotors (out of 12) and set up the rotor pins that determine the stepping motion of the rotors relative to one another. Then you place the rotors in the machine in a specific order from right to left, and set each rotor in a specific starting position. Finally, you set each of the 41 modificator switches to a previously determined position. To decrypt the message, those same rotors and settings, along with those of the modificator, must be re-created in the receiver's identical machine. All of these positions, wirings, and settings of the rotors and of the modificator are collectively known as the key.
The HX-63 includes, in addition to the hand crank, a nickel-cadmium battery to run the rotor circuit and printer if no mains power is available. A 12-volt DC linear power supply runs the motor and printer and charges the battery. The precision 12-volt motor runs continuously, driving the rotors and the printer shaft through a reduction gear and a clutch. Pressing a key on the keyboard releases a mechanical stop, so the gear drive propels the machine through a single cycle, turning the shaft, which advances the rotors and prints a character.
The printer has two embossed alphabet wheels, which rotate on each keystroke and are stopped at the desired letter by four solenoids and ratchet mechanisms. Fed by output from the rotor bank and keyboard, mechanical shaft encoders sense the position of the alphabet printing wheels and stop the rotation at the required letter. Each alphabet wheel has its own encoder. One set prints the input on the left half of the paper tape; the other prints the output on the right side of the tape. After an alphabet wheel is stopped, a cam releases a print hammer, which strikes the paper tape against the embossed letter. At the last step the motor advances the paper tape, completing the cycle, and the machine is ready for the next letter.
As I began restoring the HX-63, I quickly realized the scope of the challenge. The plastic gears and rubber parts had deteriorated, to the point where the mechanical stress of motor-driven operation could easily destroy them. Replacement parts don't exist, so I had to build such parts myself.
After cleaning and lubricating the machine, I struck a few keys on the keyboard. I was delighted to see that all nine cipher rotors turned and the machine printed a few characters on the paper tape. But the printout was intermittently blank and distorted. I replaced the corroded nickel-cadmium battery and rewired the power transformer, then gradually applied AC power. To my amazement, the motor, rotors, and the printer worked for a few keystrokes. But suddenly there was a crash of gnashing gears, and broken plastic bits flew out of the machine. Printing stopped altogether, and my heartbeat nearly did too.
I decided to disassemble the HX-63 into modules: The rotor bank lifted off, then the printer. The base contains the keyboard, power supply, and controls. Deep inside the printer were four plastic “snubbers," which cushion and position the levers that stop the ratchet wheels at the indicated letter. These snubbers had disintegrated. Also, the foam disks that ink the alphabet wheels were decomposing, and gooey bits were clogging the alphabet wheels.
I made some happy, serendipitous finds. To rebuild the broken printer parts, I needed a dense rubber tube. I discovered that a widely available neoprene vacuum hose worked perfectly. Using a drill press and a steel rod as a mandrel, I cut the hose into precise, 10-millimeter sections. But the space deep within the printer, where the plastic snubbers are supposed to be, was blocked by many shafts and levers, which seemed too risky to remove and replace. So I used right-angle long-nosed pliers and dental tools to maneuver the new snubbers under the mechanism. After hours of deft surgery, I managed to install the snubbers.
WHEELS WITHIN WHEELS: WHY THE HX-63 IS SO SECURE
The HX-63 has nine rotors
and also uses a technique
called reinjection. Each
rotor has a set of conductors
that connect each and
every electrical contact
on one side of the rotor
with a different contact on
the other side. For every
rotor the pattern of these
connections is unique. When
the operator strikes a key on
the keyboard, representing
one of 26 letters, current
travels through the set of
nine rotors twice, once in
each direction, and then
through a separate set of
15 rotor contacts at least
two times. This reinjection
technique greatly increases
the complexity of the cipher.
Chris Philpot; SOURCE: JON D. PAUL
The ink wheels were made of an unusual porous foam. I tested many replacement materials, settling finally on a dense blue foam cylinder. Alas, it had a smooth, closed-cell surface that would not absorb ink, so I abraded the surface with rough sandpaper.
After a few more such fixes, I faced just one more snafu: a bad paper-tape jam. I had loaded a new roll of paper tape, but I did not realize that this roll had a slightly smaller core. The tape seized, tore, and jammed under the alphabet wheels, deeply buried and inaccessible. I was stymied—but then made a wonderful discovery. The HX-63 came with thin stainless-steel strips with serrated edges designed specifically to extract jammed paper tape. I finally cleared the jam, and the restoration was complete.
One of the reasons why the HX-63 was so fiendishly secure was a technique called reinjection, which increased its security exponentially. Rotors typically have a position for each letter of the alphabet they're designed to encrypt. So a typical rotor for English would have 26 positions. But the HX-63's rotors have 41 positions. That's because reinjection (also called reentry) uses extra circuit paths beyond those for the letters of the alphabet. In the HX-63, there are 15 additional paths.
Here's how reinjection worked in the HX-63. In encryption mode, current travels in one direction through all the rotors, each introducing a unique permutation. After exiting the last rotor, the current loops back through that same rotor to travel back through all the rotors in the opposite direction. However, as the current travels back through the rotors, it follows a different route, through the 15 additional circuit paths set aside for this purpose. The exact path depends not only on the wiring of the rotors but also on the positions of the 41 modificators. So the total number of possible circuit configurations is 26! x 15!, which equals about 5.2 x 10
38. And each of the nine rotors' internal connections can be rewired in 26! different ways. In addition, the incrementing of the rotors is controlled by a series of 41 mechanical pins. Put it all together and the total number of different key combinations is around 10600.
Such a complex cipher was not only unbreakable in the 1960s, it would be extremely difficult to crack even today. Reinjection was first used on the NSA's KL-7 rotor machine. The technique was invented during WW II by Albert W. Small, at the U.S. Army's Signal Intelligence Service. It was the subject of a secret patent that Small filed in 1944 and that was finally granted in 1961 (No.
2,984,700).
Meanwhile, in 1953, Hagelin applied for a U.S. patent for the technique, which he intended to use in what became the HX-63. Perhaps surprisingly, given that the technique was already the subject of a patent application by Small, Hagelin was granted his patent in 1957 (No.
2,802,047). Friedman, for his part, had been alarmed all along by Hagelin's use of reinjection, because the technique had been used in a whole series of vitally important U.S. cipher machines, and because it was a great threat to the NSA's ability to listen to government and military message traffic at will.
The series of meetings between Friedman and Hagelin that resulted in the cancellation of the HX-63 was mentioned in a 1977 biography of Friedman,
The Man Who Broke Purple, by Ronald Clark, and it was further detailed in 2014 through a disclosure by the NSA's
William F. Friedman Collection.
After a career as an electrical engineer and inventor, author Jon D. Paul now researches, writes, and lectures on the history of digital technology, especially encryption. In the 1970s he began collecting vintage electronic instruments, such as the Tektronix oscilloscopes and Hewlett-Packard spectrum analyzers seen here.
Peter Adams
The revelation of Crypto AG's secret deals with U.S. intelligence may have caused a bitter scandal, but viewed from another angle, Rubicon was also one of the
most successful espionage operations in history—and a forerunner of modern backdoors. Nowadays, it's not just intelligence agencies that are exploiting backdoors and eavesdropping on “secure" messages and transactions. Windows 10's “telemetry" function continuously monitors a user's activity and data. Nor are Apple Macs safe. Malware that allowed attackers to take control of a Mac has circulated from time to time; a notable example was Backdoor.MAC.Eleanor, around 2016. And in late 2020, the cybersecurity company FireEye disclosed that malware had opened up a backdoor in the SolarWinds Orion platform, used in supply-chain and government servers. The malware, called SUNBURST, was the first of a series of malware attacks on Orion. The full extent of the damage is still unknown.
The HX-63 machine I restored now works about as well as it did in 1963. I have yet to tire of the teletype-like motor sound and the clack-clack of the keyboard. Although I never realized my adolescent dream of being a secret agent, I am delighted by this little glimmer of that long-ago, glamorous world.
And there's even a postscript. I recently discovered that my contact at Crypto AG, whom I'll call “C," was also a security officer at the Swiss intelligence agencies. And so for decades, while working at the top levels of Crypto AG, “C" was a back channel to the CIA and Swiss intelligence agencies, and even had a CIA code name. My wry old Swiss friend had known everything all along!
This article appears in the September 2021 print issue as “The Last Rotor Machine."
To Probe Further
The Crypto AG affair was described in a pair of Swedish books. One of them was
Borisprojektet : århundradets största spionkupp : NSA och ett svensk snille lurade en hel värld [translation: The Boris Project: The Biggest Spy Coup of the Century: NSA and a Swedish genius cheated an entire world], 2016, Sixten Svensson, Vaktelförlag, ISBN 978-91-982180-8-4.
Also, in 2020, Swiss editor and author Res Strehle published
Verschlüsselt: Der Fall Hans Bühler [translation: Encrypted: The Hans Bühler Case], and later Operation Crypto. Die Schweiz im Dienst von CIA und BND [Operation Crypto: Switzerland in the Service of the CIA and BND].
In the center of the cast-aluminum base of the HX-63 cipher machine is a precision Swiss-made direct-current gear motor. Also visible is the power supply [lower right] and the function switch [left], which is used to select the operating mode—for example, encryption or decryption.Peter Adams
A total of 12 different rotors are available for the HX-63, of which nine are used at any one time. Current flows into one of 41 gold-plated contacts on the smaller-diameter side of the rotor, through a conductor inside the rotor, out through a gold-plated contact on the other side, and then into the next rotor. The incrementing of each rotor is programmed by setting pins, which are just visible in the horizontal rotor.
Peter Adams
William F. Friedman [top] dominated U.S. cryptology for over 40 years and was the first chief cryptologist of the U.S. National Security Agency. His friend Boris Hagelin [bottom], a brilliant Swedish inventor and entrepreneur, founded Crypto AG in 1952 in Zug, Switzerland, and built it into the world's largest cipher-machine company.TOP, U.S. ARMY; BOTTOM: CRYPTO AG
Chris Philpot; SOURCE: JON D. PAUL
While encrypting or decrypting a message, the HX-63 prints both the original and the encrypted message on paper tape. The blue wheels are made of an absorbent foam that soaks up ink and applies it to the embossed print wheels.Peter Adams
Beneath the nine rotors on the HX-63 are nine keys that unlock each rotor to set the initial rotor position before starting a message. That initial position is an important component of the cryptographic key.Peter Adams
Chris Philpot; SOURCE: JON D. PAUL
After a career as an electrical engineer and inventor, author Jon D. Paul now researches, writes, and lectures on the history of digital technology, especially encryption. In the 1970s he began collecting vintage electronic instruments, such as the Tektronix oscilloscopes and Hewlett-Packard spectrum analyzers seen here.
Peter Adams